Image display apparatus, image display method, and computer program

ABSTRACT

An average color bar indicating the overall imaging period of images taken in time sequence by a capsule endoscope is displayed. A list of checked images in the entire taken images is displayed in a checked-image display field, computation is made to what time during an observation period each checked image corresponds is computed, and a mark is displayed with a scale of the average color bar by a number corresponding to each checked image on the average color bar.

RELATED APPLICATION

This application is a divisional of U.S. Ser. No. 10/830,790, filed Apr.23, 2004, the entire contents of which are incorporated herein byreference, which claims priority from Japanese Application Number JP2003-122804 filed on Apr. 25, 2003.

BACKGROUND OF THE INVENTION

1) Field of the Invention

The present invention relates to an image display apparatus, an imagedisplay method, and an image display program.

2) Description of the Related Art

Recently, swallowable capsule endoscopes have been produced as a type ofendoscopes. The capsule endoscopes are provided with an imagingcapability and a radio capability. A capsule endoscope is configured tosequentially take images of organs such as the stomach and the smallintestine within an observation period from the time it has beenswallowed through the mouth of a patient for observation (examination)to its natural excretion from the human body (see Japanese PatentApplication Laid-open No. H11-225996 Publication).

During the observation period, image data taken in a body by the capsuleendoscope is sequentially transmitted outside through radiocommunication and is stored in a memory. Since a patient carries arounda receiver having a radio communication capability and a memorycapability, the patient can freely perform normal actions during theobservation period from swallowing of the capsule endoscope to itsexcretion. After observation, a doctor or a nurse can display the imagesof organs on a display based on the image data stored in the memory anduse it to make a diagnosis.

As the above type of capsule endoscope, “M2A (registered trademark)” byGiven Imaging Ltd. of Israel, and “NORIKA (registered trademark)” by RFSYSTEM lab. of Japan are presently available, and they have already cometo practical applications.

However, unlike an ordinary endoscope, the capsule endoscope describedabove takes images of each organ within a period from the time a subjectswallows to its natural excretion, meaning an extended period ofobservation (examination), for example, more than ten hours. Therefore,the number of images to be taken in time sequence is correspondinglyhuge.

At the stage of diagnosis or the like, no particular consideration isgiven to improving the ability to retrieve a desired image from the vastamount of images taken over a long period of time, or providing adisplay screen allowing easy recognition of what time in the overallimaging period the displayed image was taken, of which organ is beingshown, and the like.

SUMMARY OF THE INVENTION

It is an object of the present invention to solve at least the problemsin the conventional technology.

The image display apparatus according to one aspect of the presentinvention includes an input unit that inputs image data taken in timesequence by an in-vivo imaging device, a scale display control unit thatcontrols to display a scale indicating an overall imaging period ofinput image data taken in time sequence and input by the input unit, acolor information detecting unit that detects color information of ascreen of the image data input by the input unit, a color displaycontrol unit that controls to display a color corresponding to the colorinformation detected by the color information detecting unit at atime-corresponding position on the scale, an image display control unitthat controls to display an image corresponding to the image data inputby the input unit, an image designation unit that designates the imagesubjected to be displayed by the image display control unit, and anindex display control unit that controls to display, on the scale, anindex indicating a position corresponding to an imaging time of theimage designated by the image designation unit.

The image display method according to another aspect of the presentinvention includes inputting image data taken in time sequence by anin-vivo imaging device, displaying a scale indicating an overall imagingperiod of input image data taken in time sequence and input by the inputunit, detecting color information of a screen of the image data input bythe input unit, displaying a color corresponding to the colorinformation detected by the color information detecting unit at atime-corresponding position on the scale, displaying an imagecorresponding to the image data input by the input unit, designating theimage subjected to be displayed by the image display control unit, anddisplaying, on the scale, an index indicating a position correspondingto an imaging time of the image designated by the image designationunit.

The image display program according to still another aspect of thepresent invention realizes the method according to the above aspect on acomputer.

The other objects, features, and advantages of the present invention arespecifically set forth in or will become apparent from the followingdetailed description of the invention when read in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of a capsule endoscope according to an embodimentof the present invention;

FIG. 2 is a schematic of a capsule endoscope system according to theembodiment;

FIG. 3 is a block diagram of an example of the capsule endoscope systemaccording to the embodiment;

FIG. 4A and FIG. 4B are schematics of an example of screen transition(screen 1 and 2) associated with the observation procedures according tothe embodiment;

FIG. 5A and FIG. 5B are schematics of an example of screen transition(screen 3 and 4) associated with the observation procedures according tothe embodiment;

FIG. 6A to FIG. 6C are schematics of an example of screen transition(screen for acquisition of data 1 to 3) associated with the observationprocedures according to the embodiment;

FIG. 7 is a schematic of an example of screen transition associated withthe diagnosis procedures according to the embodiment;

FIG. 8 is a schematic of an example of screen transition associated withthe diagnosis procedures according to the embodiment;

FIG. 9 is a flowchart of the operation for average color bar displayaccording to the embodiment;

FIG. 10 is a schematic of an example of a display screen associated witha diagnosis process according to a modification of the embodiment;

FIG. 11 is Graphs for illustrating the principle of automaticdiscrimination of organ names according to the modification of theembodiment;

FIG. 12 is a flowchart of the procedures of discriminating the organnames according to the modification of the embodiment;

FIG. 13 is a graph for illustrating an example of application of themodification shown in FIG. 11;

FIG. 14 is a schematic of an example of screen transition associatedwith the diagnosis procedures according to the embodiment; and

FIG. 15 is a flowchart of an operation for displaying the imaging timeof a designated image according to the embodiment.

DETAILED DESCRIPTION

Exemplary embodiments of an image display apparatus, an image displaymethod, and a computer program according to the present invention aredescribed below in detail with reference to the accompanying drawings.

FIG. 1 is a schematic of a capsule endoscope according to an embodimentof the present invention. A capsule endoscope 10 includes an imagingunit 111 that can take the internal image of a celom, illumination units112 a and 112 b that illuminate the interior of the celom, a powersupply unit 13 that supplies them with power, and a capsule housing 14that has at least the imaging unit 111, the illumination units 112 andthe power supply unit 13 disposed inside.

The capsule housing 14 according to the present embodiment includes adistal-end cover 120 that covers the imaging unit 111 and theillumination units 112 a, 112 b, and a capsule body 122 that is providedin a water-proof state with respect to the distal-end cover 120 via aseal member 121 and has the imaging unit 111, etc. disposed therein. Arear-end cover 123 may be provided as separate from the capsule body 122as needed. Although the rear-end cover 123 is provided integrally withthe capsule body and has a flat shape in the present embodiment, theshape is not limited and may be, for example, a dome shape.

The distal-end cover 120 may clearly separate an illumination window 120a, that transmits illumination light L from the illumination unit 112 a,112 b, and an imaging window 120 b, that performs imaging in theillumination range, from each other. In the present embodiment, theentire distal-end cover 120 is transparent and the areas of theillumination window 120 a and the imaging window 120 b partly overlapeach other.

The imaging unit 111 is provided on an imaging board 124 with asolid-state imaging device 125 formed of, for example, a CCD, whichperforms imaging in the range that is illuminated with the illuminationlight L from the illumination unit 112 a, 112 b, and an image forminglens 126 that includes a fixed lens 126 a and a movable lens 126 b, andforms the image of a subject to the solid-state imaging device 125, andexecutes sharp image forming with a focus adjusting unit 128 with afixed frame 128 a that secures the fixed lens 126 a and a movable frame128 b, which secures the movable lens 126 b. In the present invention,the imaging unit 111 is not limited to the CCD, but an imaging unit suchas CMOS, may be used.

The illumination units 112 a, 112 b are provided on an illuminationboard 130 and are comprised of, for example, a light-emitting diode(LED), and a plurality of illumination units 112 a, 112 b (four in thepresent embodiment as one example) are laid out around the image forminglens 126 that constitutes the imaging unit 111. In the presentinvention, the illumination units 112 a, 112 b are not limited to theLED but other illumination units may be used as well.

The power supply unit 13 is provided on a power supply board 132provided with an internal switch 131 and uses, for example, a buttontype battery as a power supply 133. While a silver oxide cell, forexample, is used as the battery in the present invention, the inventionis not limited to it and may use a chargeable battery, a dynamo typebattery or the like.

Although one that can perform an ON operation by, for example, theoppositional action of magnets is used as the internal switch 131, thepresent invention is not limited to this type and other switch units canbe also exemplified.

In the present embodiment, besides the individual units described above,a radio unit 142 comprising an antenna or the like for radiocommunication with outside is provided on a radio board 141 andcommunication with outside is carried out as needed.

A signal processing/control unit 143 for processing or controlling theindividual units is provided on an imaging board 124 and executesvarious processes in the capsule endoscope 10.

The signal processing/control unit 143 is comprised of a video signalprocessing function for image data generation, a transmission signalgenerating function that performs mixing of a video signal and a syncsignal, affixing of an error correction code, etc., a modulationfunction that performs conversion to, for example, a PSK, MSK, GMSK,QMSK, ASK, AM, or FM system in cooperation with a modulator, a powersupply control function that controls power supply with ON-OFF of aswitch, driver circuits such as an LED driver circuit, a timinggenerator (TG) function that controls the number of imaging shots, and amemory function that stores various data, such as parameters for a lineframe. The signal processing/control unit 143 executes various signalprocesses/controls.

The video signal processing function performs processes, such as imagedata correction (e.g., white balance (WB) correction, γ correction,color processing, correlation double sampling (CDS), and automatic gaincontrol (AGC)), and analog-digital conversion (ADC) and an auto exposurefunction (AE), in addition to, for example, image data generation.

Besides the communication unit 142, for example, information collectingunits, such as various sensors, a chemical releasing unit that releaseschemicals, a tissue collecting unit that cuts tissues in a celom andcollects them, etc. may be disposed in the capsule endoscope 10 asneeded.

FIG. 2 is a schematic of a capsule endoscope system according to theembodiment. At the time of performing examination using the capsuleendoscope 10, the capsule endoscope system as shown in FIG. 2 is used.

The capsule endoscope system according to the present embodimentcomprises the capsule endoscope 10 and its package 50, a jacket 3 that apatient or a subject 2 wears, a receiver 4 attachable to/detachable fromthe jacket 3, a work station 5, a CF (compact flash (registeredtrademark)) memory reader/writer 6, a label printer 7, a database 8, anda network 9, as shown in FIG. 2, for example.

The jacket 3 is provided with antennas 31, 32, 33, and 34 that catchradio waves of taken images to be sent from the radio unit 142 of thecapsule endoscope 10 so that the jacket 3 can communicate with thereceiver 4 wirelessly or by a cable. The number of antennas is notparticularly limited to four but should be plural, so that radio wavesaccording to positions of the capsule endoscope 10 moved can be receivedproperly.

The receiver 4 is provided with an antenna 41 that is used when directlyreceiving taken images through radio waves, a display unit 42 thatdisplays information necessary for observation (examination) and aninput unit 43 that inputs information necessary for observation(examination). A CF memory 44 that stores received taken image data canbe detachably attached to the receiver 4. Further, the receiver 4 isprovided with a power supply unit 45 capable of supplying power even atthe time of portable usage and a signal processing/control unit 46 thatperforms processes needed for observation (examination). As the powersupply unit 45, for example, a dry cell, Li ion secondary battery, andNi hydrogen battery can be exemplified and a chargeable type may also beused.

The work station 5 has a processing function for performing a diagnosisbased on images of organs or the like in a patient, taken by the capsuleendoscope 10 by a doctor or a nurse. This work station 5 has interfaces,though not shown, which connect to the receiver 4, the CF memoryreader/writer 6, and the label printer 7 in a communicable manner andexecutes read/write of the CF memory 44, chart printing, etc.

The work station 5 has a communication function for connecting to thenetwork 9 and stores doctor results of a patient into the database 8 viathe network 9. Further, the work station 5 has a display unit 51, andreceives taken image data of inside a patient from the receiver 4 anddisplays the images of organs or the like on the display unit 51.

As the capsule endoscope 10 is taken out of the package 50 and isswallowed by the subject 2 through the mouth, prior to initiationexamination, it passes through the esophagus, moves inside the celom byperistalsis of the digestive tracts and takes images inside the celomone after another.

The radio waves of taken images are output via the radio unit 142 asneeded or for the imaging results and are caught by the antennas 31, 32,33, and 34 of the jacket 3. A signal from the antenna the intensity ofwhose received radio waves is high is sent to the receiver 4 outside.

In the receiver 4, taken image data received one after another is storedin the CF memory 44. The receiver 4 is not synchronized with the startof imaging of the capsule endoscope 10 and the initiation of receptionand end of reception are controlled by manipulation of the input unit43. The taken image data may be still picture data taken by pluralframes per second for dynamic display or ordinary moving picture data.

When observation (examination) of the subject 2 by the capsule endoscope10 is finished, the taken image data stored in the CF memory 44 istransferred to the work station 51 via a cable. The work station 5memorizes the transferred taken image data in association withindividual patients.

The taken image data inside the celom taken by the capsule endoscope 10and stored in the receiver 4 in this manner is displayed by the displayunit 51 of the work station 5. Accordingly, acquisition of effectivedata for physiological study and diagnosis of lesion can be carried outover the entire digestive tracts of a human body including the deep bodyportion (small intestine, etc.) that cannot be reached by an ultrasonicprobe, endoscope, etc.

FIG. 3 is a block diagram of an example of the capsule endoscope systemaccording to the embodiment. The description is given on only theessential structures of the individual units.

The capsule endoscope 10 has the structure to take the image of aninternal target (organs, etc.) with the imaging unit 111 from reflectionof light illuminated from the illumination units 112 a and 112 b andsend the taken image from the radio unit 142 in the form of a radiosignal.

The jacket 3 has a structure such that a selector 35 is connected to thefour antennas 31, 32, 33, 34, and an I/F 36 to which a cable to connectto the receiver 4 is connected to the selector 35. The jacket 3 receivesradio signals sent from the capsule endoscope 10 at the four antennas31, 32, 33, and 34, select a received signal according to the radio waveintensity by the selector 35 and is transferred to the receiver 4 viathe I/F 36. The jacket 3 is not provided with a large-capacity memoryand taken images received via the antennas 31, 32, 33, and 34 aretransferred one after another to the receiver 4 at the subsequent stage.

The receiver 4 has, as the internal structure, an I/F 45 forcommunication to the I/F 36 of the jacket 3 via a cable, a CPU 46 thatcontrols the entire receiver 4 according to a program preparedbeforehand, a CF memory I/F 47 that performs data communication with theattached CF memory 44, and an I/F 48 that performs communication withthe work station 5 by a cable.

To secure the state of being capable of receiving taken images from thejacket 3 at any time, the receiver 4 is always attached to the subject 2during observation of inside a body by the capsule endoscope 10. Duringobservation, therefore, taken images are received one after another fromthe jacket 3 and the received images are stored in the CF memory 44 viathe CF memory I/F 47 one after another. During observation, the receiver4 is not connected to the work station 4 and the subject 2 is notrestricted in a hospital or the like and can move freely.

The CF memory reader/writer 6 has, as the internal structure, a CPU 61that controls the entire reader/writer according to a program preparedbeforehand, a CF memory I/F 62 that performs data communication with theattached CF memory 44, and an I/F 63 that performs communication withthe work station 5 by a cable.

The CF memory reader/writer 6 is attached with the CF memory 44 and isconnected to the work station 5 via the I/F 63, performs formatting oftaken information for diagnosis according to the present embodiment withrespect to the CF memory 44 or reads stored taken image data from the CFmemory 44 and transfers the data to the work station 5. The taken imagedata here is in the form of JPEG or the like.

According to the present embodiment, it is possible to arbitrarilyselect direct transfer of taken image data to the work station 5 fromthe receiver 4 or moving the CF memory 44 to the CF memory reader/writer6 to transfer taken image data to the work station 5.

The work station 5 has the display unit 51 that displays images oforgans, etc. according to the present embodiment, an I/F 52 that managescommunication with the I/F 48 of the receiver 4 via a cable and the I/F63 of the CF memory reader/writer 6 via a cable, a large-capacity memory53 that stores data to be handled in various processes, a CPU 54 thatcontrols the entire work station 5 according to a program preparedbeforehand, an input unit 55 that inputs various kinds of operations andan output unit 56 that is connected to the label printer 7 or thedatabase 8 or other printers over the network 9 for performing variouskinds of output processes.

When the observation period ends and the receiver 4 is connected to thework station 5 in a communicable manner, taken image data stored in theCF memory 44 is transferred from the receiver 4 to the work station 5and stored in the memory 53. In the work station 5, taken images fromthe capsule endoscope 10 according to the present embodiment, thedisplay of an average color slider to be discussed later, the locus ofthe capsule endoscope 10, etc. are displayed at the time of a diagnosis.The diagnosis results are output as a chart from the printer and storedin the database 8 patient by patient.

FIG. 4A and FIG. 4B, FIG. 5A and FIG. 5B, and FIG. 6A to FIG. 6C areschematics of an example of screen transition associated with theobservation procedures according to the present embodiment. FIG. 7 andFIG. 8 are schematics of an example of screen transition associated withthe diagnosis procedures according to the present embodiment. FIG. 9 isa flowchart of the operation for average color bar display according tothe embodiment. A program for displaying an average color slider isdirectly installed from a recording medium such as CD-ROM or isdownloaded from outside such as a network, then installed and stored inthe memory 53 of the work station 5 as its storage scheme.

First, a doctor (or a nurse) formats the CF memory 44 using the workstation 5 and the CF memory reader/writer 6. In this case, as proceduresprior to observation, the CF memory 44 is inserted into the CF memoryreader/writer 6 and a guidance screen prompting connection of the CFmemory reader/writer 6 to the work station 5 is displayed on the displayunit 51 of the work station 5 (FIG. 4A). When the doctor performs a menuoperation for “NEXT”, the process proceeds to the next guidance screendisplay. It is assumed that the doctor has prepared according to theguidance at this time. If the preparation is inadequate and the menuoperation for “NEXT” is done in that state, a message of non-insertionof the CF memory, non-connection of the CF memory reader/writer or thelike may be displayed.

The next guidance screen displays a guidance screen prompting entry ofdiagnosis information and patient information (FIG. 4B). As thediagnosis information, there are input items of, for example, a hospitalname, the name of capsule-administering doctor (nurse), the date/time ofcapsule administration, a capsule serial number and a receiver serialnumber. As the patient information, there are input items of, forexample, a patient ID, the name of a patient, gender of the patient, theage of the patient and the birth date of the patient. When the inputoperation for various input items is completed and the menu operationfor “NEXT” is done, a confirmation screen for the entered items isdisplayed (FIG. 5A). The screen may go back to the previous screenthrough a menu operation for “BACK”.

As the next guidance screen (FIG. 5A) shows a confirmation of the itemsentered on the previous screen and the doctor further performs the menuoperation for “NEXT”, it is considered that nothing is wrong about theinput information and the display screen goes to the next screen (FIG.5B). At this time, information on the input items is written in the CFmemory 44. When the menu operation for “BACK” is done, the items enteredpreviously can be corrected.

The next guidance screen (FIG. 5B) shows a message of an instruction toremove the CF memory 44, an instruction to put labels having necessaryID information printed according to the input items confirmation of theitems entered on the previous screen to the receiver 4 and the CF memory44, and an instruction to insert the CF memory 44 into the receiver 4.When the doctor performs a menu operation for “COMPLETED”, preparationbefore administration of the capsule endoscope 10 into the subject iscompleted.

Then, the administration of the capsule endoscope 10 into the subject 10is completed, observation of the interior of the body is started andstorage of taken image data into the CF memory 44 is started by theoperation of the receiver 4. When the observation period ends andstorage into the CF memory 44 is finished, the doctor receives guidancefrom the work station 5 again.

First, the CF memory 44 is removed from the receiver 4 and a guidancescreen prompting insertion of the CF memory reader/writer 6 is displayed(FIG. 6A). After preparation takes places according to the message, whenthe doctor performs the menu operation for “NEXT”, the display screengoes to the next (FIG. 6B).

In the next guidance screen (FIG. 6B), the diagnosis information andpatient information recorded in the CF memory 44 are read from thememory and displayed. The information of the displayed contents, i.e.,information (taken image data, etc.) acquired through observation isacquired by the work station 5.

When the doctor performs the menu operation for “NEXT” upon completionof acquisition of the information in that manner, a process of acquiringdata from the CF memory 44 is carried out. When the data acquisitionprocess is finished, a guidance screen prompting completion of dataacquisition from the CF memory 44, removal of the CF memory 44 from theCF memory reader/writer 6 and instruction for initiation of diagnosis isdisplayed (FIG. 6C). When the doctor performs the menu operation for“COMPLETED”, a sequence of guidance associated with the observationprocedures is completed.

In the transition of a series of screens, there are icons of CANCEL andHELP that the doctor can arbitrarily select and operate. When the CANCELis operated, the inputs so far are initialized.

At the stage of the diagnosis process, first, a list of diagnosisinformation and patient information of individual patients saved in thememory 53 of the work station 5 is displayed (FIG. 7). Accordingly, thedoctor can select on which patient diagnosis is to be done with, forexample, a cursor. The selected state has only to be given in inverteddisplay. When a menu operation for “OBSERVATION” is done with the cursorselecting state, a patient to be diagnosed is decided. With regard todiagnosed patients, affixing “DONE” on the displayed list as shown inFIG. 7 can ensure an easy confirmation of whether a diagnosis has beenmade.

As a patient to be diagnosed is decided in this manner, a diagnosisprocedure screen is displayed as shown in FIG. 8. This diagnosisprocedure screen shows information necessary for diagnosis. 501 and 502are respectively patient information and diagnosis information of theassociated patient, and 503 is an image display field illustrating oneof taken images. 504A shows a checked-image display field giving a listof taken images of interest that have been arbitrarily checked(selected) by a doctor by operating a software-based check button CHK.

505 shows a 3D (three dimensional) position display field showing animaging position (position inside a body) of the taken image, displayedin the image display field 503, in a 3D manner, 506 shows a playbackoperation field 506 for performing a playback operation for a takenimage to be displayed in the image display field 503, and 507 shows anaverage color bar colored in time sequence with average colors accordingto the organs for taken images from the start point of reception by thereceiver to the end point of reception. The average color bar 507 servesas a scale indicating the passing time during the observation period.The display screen further displays individual menus for “HELP”, “BACK”,“CANCEL”, and “END DIAGNOSIS/PRINT CHART”.

The average color bar 507 is average colors acquired from the individualframes of a taken image and colored in time sequence using thecharacteristics of colors different from one organ to another. In theaverage color bar 507, therefore, the average color of a taken imagewhen the capsule endoscope 10 is moving according to regions of eachorgan becomes nearly uniform. Even if an image taken while movement inthe same organ contains noise, nearly a uniform color for each organ canbe acquired by obtaining the average color of a single screen frame byframe.

In the average color bar 507, a slider S is shown movable in thedirection of the time axis. The slider S serves as an index to indicatethe position of a taken image to be displayed in the image display field503, at a position on the average color bar 507. Therefore,moving/display control of the slider S is carried out according to theoperation of the playback operation field 506.

The movement of the slider S on the average color bar 507 and changingof the taken image to be displayed in the image display field 503 aresynchronized. That is, a software-based FRAME PLAYBACK button, PLAYBACKbutton, and FAST PLAYBACK (FP) button for operations in the forwardplayback direction along the time-sequential direction and asoftware-based REVERSE FRAME PLAYBACK button, REVERSE PLAYBACK button,and FAST REVERSE PLAYBACK (FR) button for operations in the reverseplayback direction along the time-sequential direction are displayed andcontrolled. Further, a STOP button is displayed and controlled in theplayback operation field 506.

When a doctor clicks the PLAYBACK button with a mouse (not shown) byoperating the input unit 55, an image based on taken image data isdisplayed in the image display field 503 in time sequence in the forwardplayback direction. When the FRAME PLAYBACK button is clicked, a nextimage in the forward playback direction is displayed, and when the FASTPLAYBACK button is clicked, images are reproduced and displayed fasterthan the playback done by the PLAYBACK button in the forward playbackdirection. When the STOP button is clicked during playback or duringfast playback, changing of the displayed image is stopped while an imageat the time the clicking was made is displayed.

When the doctor clicks the REVERSE PLAYBACK button with the mouse (notshown) by operating the input unit 55, an image based on taken imagedata is displayed in the image display field 503 in the reverse playbackdirection with respect to the time-sequential direction. When theREVERSE FRAME PLAYBACK button is clicked, an image previous by one inthe forward playback direction is displayed, and when the FAST REVERSEPLAYBACK button is clicked, images are reproduced and displayed fasterthan the playback done by the REVERSE PLAYBACK button in the reverseplayback direction. When the STOP button is clicked during reverseplayback or during fast reverse playback, changing of the displayedimage is stopped while an image at the time the clicking was made isdisplayed.

When a diseased part like a bleeding part is found, or the like at thetime of image playback or reverse playback in the image display field503, a checked image distinguished from other images can be extracted atthe doctor's discretion. When such checking is desired, the doctoroperates the check button CHK. The checked image is additionallydisplayed as a thumbnail image in the checked-image display field 504A.Due to the restriction of the display area, the checked-image displayfield 504A can display up to a predetermined number of images. In thepresent embodiment, as shown in FIG. 8, for example, up to five imagescan be displayed and for other checked images, display images areswitched by scrolling.

As the average color bar 507 is segmented by the average colorsaccording to the types of the organs, the doctor can intuitively andquickly move the display image to the position of the taken imageassociated with the desired organ referring to the average color bar507. At this time, the slider S of the average color bar 507 is moved byusing the mouse (not shown). As the slider S is operated to move on theaverage color bar 507, a process of sequentially changing the image tothe one at the position indicated by the slider S following the movementis executed in the image display field 503.

In the present embodiment, when the doctor finds a bleeding part fromthe display image, a flag as a bleeding part can be affixed to eachtaken image. In this case, though not shown, a sub menu is displayedwith the current state displayed in the image display field 503 tomanually set the flag of the bleeding part. Accordingly, display can bemade in association with the positions on the average color bar 507,such as bleeding parts V1, V2, as shown in FIG. 8, for example.

A bleeding part can be automatically extracted through image processing,in which case an AUTO-RETRIEVE BLEEDING PART button as indicated by 508is operated. The operation of the AUTO-RETRIEVE BLEEDING PART button 508may be done for the image currently displayed in the image display field503 or for all the images. When it is found in automatic retrieval, aflag is put in association with each image as done in the case of manualoperation.

The diagnosis by a doctor can be terminated by a menu operation for “ENDDIAGNOSIS/PRINT CHART”. The diagnosis results are made into a chart andprinted through a printer (not shown) from the work station 5 or via thedatabase 8.

In the display of the average color bar 507, a process is executed asshown in FIG. 9. That is, when a patient to be diagnosed is decided froma list shown in FIG. 7, a file of imaging information corresponding tothat patient is designed. Then, one frame of image files is read fromthe memory 53 and opened (step S1), and the average color of the takenimages frame by frame is measured (step S2).

When the average color is measured and average color data is acquired,the average color data for the first frame is stored in the memory 53(step S3). Then, a processed image file is closed and an image filelocated next in time sequence is read out and opened, and a similarprocess is repeatedly executed thereafter (NO route of step S5).

When the average colors for all the imaging information of the patientto be diagnosed are obtained (step S5), the average color bar 507 isdisplayed and controlled as shown in FIG. 8 using the average color datastored in the memory 53 (step S6). In this manner, the display of theaverage color bar 6 is completed. At this time, the initial position ofthe slider S is the left end (start position) of the average color bar507 but is not restrictive.

Because the amount of the imaging information including taken image datais huge, it is unnecessary to open all the image files and acquire theaverage colors for all the frames, and the average color may be acquiredwhile efficiently thinning several frames. Although the acquired averagecolor itself is displayed on the average color bar 507 in the presentembodiment, it is not restrictive and a color corresponding to thisaverage color has only to be displayed on the average color bar 507.

According to the present embodiment, as described above, a scaleindicating the overall imaging period of input image data taken in timesequence by the capsule endoscope (internal imaging device) isdisplayed, a movable slider is shown on the scale, an image at theimaging time corresponding to the position of the slider is displayed inresponse to the movement of the slider on the scale, and a colorcorresponding to average color information for one screen of input imagedata is displayed at the time-associated position on the scale, so thatdistinguishing coloring is carried out according to the taken part andan organ in the body can easily be determined from the distinguishedcolors. Accordingly, the ability to retrieve the image is improved andit is possible to easily recognize the organ depicted in each image.

Although the position of an organ is identified using the average colorsarranged on the average color bar as an index in the embodimentdescribed above, the present invention is not limited to this type andan additional function of displaying the name of an organ in associationwith the average color may be provided as in a modification to bediscussed below. As the modification to be discussed below is the samein the structure and functions described above, only what is added isdiscussed.

FIG. 10 is a schematic of an example of a display screen associated witha diagnosis process according to a modification of the embodiment. FIG.11 is Graphs for illustrating the principle of automatic discriminationof organ names according to the modification of the embodiment. FIG. 12is a flowchart of the procedures of discriminating the organ namesaccording to the modification of the embodiment.

The organ names are displayed in association with each average color onthe average color bar 507. Average colors are lined on the average colorbar 507 in the order of the esophagus, the stomach, the small intestine,and the large intestine in the order of imaging done in a body by thecapsule endoscope 10 in time sequence. Therefore, the average color bar507 shows organ names 509 in the order of the esophagus, the stomach,the small intestine, and the large intestine in association with theaverage colors of the individual organs.

At the time of automatic discrimination of organ names, it is theautomatic discrimination in the ranges of organs. The level of red andthe level of blue for individual taken images at elapsed times have thecharacteristics as shown in FIG. 11. As an actual image contains a noisecomponent, it is subjected to a low-pass filter (LPF) process in thedirection of the time axis with respect to the levels of red and bluethat have the characteristics to remove noises. Then, edge portions(discoloration edges) the levels of red and blue in the direction of thetime axis after the LPF process commonly have are extracted.

In the example in FIG. 11, there are three discoloration edges, (1),(2), and (3), extracted in the above manner. Therefore, automaticdiscrimination is done such that from the positions of the discolorationedges (1), (2), and (3) in the direction of the time axis, the firstdiscoloration edge (1) is a transitional portion from the esophagus tothe stomach, (2) is a transitional portion from the stomach to the smallintestine and (3) is a transitional portion from the small intestine tothe large intestine. At this time, the order of the organ names is basedon the layout of the organs to be taken by the capsule endoscope 10 inthe direction of the time axis.

As the processing based on the principle described above, first, the redlevel and blue level are computed (step S21), the LPF process in thedirection of the time axis is performed on the red level and blue level(step S22) and the discoloration edges (1), (2), and (3) are detected(step S23). Then, automatic discrimination of the ranges of the organsis carried out from the time-associated positions of the discolorationedges (1), (2), and (3) and the organ names are displayed in associationwith the individual average colors on the average color bar 507 (stepS24).

In the above manner, a scale indicating the overall imaging period ofinput image data taken in time sequence by the capsule endoscope isdisplayed, a movable slider is shown on the scale, an image at theimaging time corresponding to the position of the slider is displayed inresponse to the movement of the slider on the scale, and organs arediscriminated based on color information for one screen of input imagedata and organ names are displayed in association with the scale, sothat organs in the body can easily be determined from the displayedorgan names. This also improves the ability to retrieve images and makesit possible to easily recognize the organ depicted in each image.

Although the ranges of the organs on the average color bar areautomatically discriminated from the discoloration edges in themodification described above, the present invention is not limited tothis type and a pH sensor may be provided in the capsule endoscope 10 sothat the ranges of the organs are specified more accurately using themeasured pH values. In this case, the pH values are measured by the pHsensor during the observation period and like taken images, the pHvalues are measured in time sequence and are stored in the receiver 4.At that time, the taken images and pH values are recorded in associationwith each other, such as coexisting in each frame (image file).

FIG. 13 is a graph for illustrating an example of application of themodification shown in FIG. 11. In the automatic discrimination with pHvalues added, as shown in FIG. 13, using the fact that the stomach is inan acidic state, an acidic part is compared with the discoloration edges(1) and (2) to discriminate the stomach part, thereby further increasingthe discrimination precision.

FIG. 14 is a schematic of an example of screen transition associatedwith the diagnosis procedures according to the embodiment. FIG. 15 is aflowchart of an operation for displaying the imaging time of adesignated image according to the embodiment. While a diagnosis by adoctor can be terminated through the menu operation for “ENDDIAGNOSIS/PRINT CHART”, further transition to the chart creatingprocedures can be made.

When the process is shifted from the display screen in FIG. 8 to thedisplay screen in FIG. 14, comments of a doctor are entered and a markindicating to which elapsed time on the average color bar 507 eachchecked image corresponds is displayed.

That is, 504B in FIG. 14 indicates a checked-image display field, setlarger than the checked-image display field 504A and provided at thelower portion of the screen. As a difference from the checked-imagedisplay field 504A, numbers (1) to (10) are given to individual takenimages and displayed. The checked-image display field 504B has the samefunction as the checked-image display field 504A.

510 is a comment input field where opinions (comments) of a doctor areinput and displayed. The results of a diagnosis by a doctor are input ascomments in the comment input field 510. 511 indicates an imaging timedisplay mark that is displayed, as a mark on the average color bar 505,indicating which taken image at which elapsed time each checked image tobe displayed in the checked-image display field 504B is. As the imagingtime display mark, a downward arrow as an index indicating the imagingtime for a checked image and the aforementioned number given to achecked image as relative display indicating the correlation with thechecked image to show the correlation with the checked image aredisplayed on the average color bar 505.

FIG. 14 displays ten checked images. In this example, average colors aredistinguished on the average color bar 507 in the order of theesophagus, the stomach, the small intestine, and the large intestine. Asapparent from the ranges of the organs of the organ names 509,therefore, a mark (1) for a checked image is present in the range of theesophagus, and marks (2), (3), and (4) for a checked image are presentin the range of the stomach. Further, marks (5), (6), (7), (8), (9), and(10) for checked images are present in the range of the small intestine.

Therefore, the presence of images checked by a doctor are identified inthe esophagus, the stomach, and the small intestine from the example inFIG. 14, and marks are displayed in association with the times at whichthe individual checked images have been taken, so that the doctor caneasily confirm at which parts of the organs the checked images have beentaken. Although the imaging time display mark is displayed on theaverage color bar 505 showing the organ names in FIG. 14, it may bedisplayed on the average color bar that does not show the organ names asin FIG. 8. Although a correlation indication (number) indicating thecorrelation with a checked image is displayed as the imaging timedisplay mark in FIG. 14, it may be an index (downward arrow) indicatingthe position of the imaging time.

The process for the above mark display is described with reference toFIG. 15. In the imaging time display of a checked image or a designatedimage, first, the date/time of creating a file of the designated imageis acquired from the memory 53 (step S31), and the time elapsed sincethe date/time of the initiation of imaging is computed (step S32). Then,a mark display as shown in FIG. 4 is controlled on the scale of theaverage color bar 507 at the position corresponding to the elapsed timeon the average color bar 507 (step S33). Thereafter, when chart printingis manipulated, outputting for the chart printing is executed.

According to the present embodiment, as described above, a scaleindicating the overall imaging period of input image data taken in timesequence by the capsule endoscope (internal imaging device) isdisplayed, a color corresponding to average color information for onescreen of input image data is displayed at a time-associated position onthe scale, an image corresponding to the input image data is displayed,and an index indicating a position corresponding to an imaging time of adesignated image is displayed, so that it is possible to visually andeasily recognize how many and in which time band designated images arepresent. As organs can easily be determined from the colorsdistinguished from one taken part from another one, it is possible toeasily recognize which part of which organ has more designated images.

Furthermore, a scale indicating the overall imaging period of inputimage data taken in time sequence by the capsule endoscope is displayed,organs are discriminated based on color information of one screen ofinput image data, the names of the discriminated organ are displayed inassociation with the scale, images corresponding to the input image dataare displayed and an index indicating the position corresponding to theimaging time of the designated image is displayed on the scale, so thatorgans in the body can easily be determined from the displayed organnames. This also makes it possible to easily recognize which part ofwhich organ has more designated images.

The present invention is not limited to the above embodiments, andvarious modifications can be made without departing from the spirit ofthe present invention.

As explained above, according to the present invention, it is possibleto provide an image display apparatus constructed in such a way that ascale indicating the overall imaging period of input image data taken intime sequence by an internal imaging device is displayed, a colorcorresponding to average color information for one screen of input imagedata is displayed at a time-associated position on the scale, an imagecorresponding to the input image data is displayed, and an indexindicating a position corresponding to an imaging time of a designatedimage is displayed, so that it is possible to visually and easilyrecognize how many and in which time band designated images are presentand easily determine organs from the colors distinguished from one takenpart from another one, thus making it possible to easily recognize whichpart of which organ has more designated images.

Furthermore, according to the present invention, it is possible toprovide an image display apparatus constructed in such a way that ascale indicating the overall imaging period of input image data taken intime sequence by an internal imaging device is displayed, organs arediscriminated based on color information of one screen of input imagedata, names of the discriminated organ are displayed in association withthe scale, images corresponding to the input image data are displayedand an index indicating the position corresponding to the imaging timeof the designated image is displayed on the scale, so that organs in thebody can easily be determined from the displayed organ names, whereby itis possible to easily recognize which part of which organ has moredesignated images.

Moreover, according to the present invention, it is possible to providean image display method configured to have steps of displaying a scaleindicating the overall imaging period of input image data taken in timesequence by an internal imaging device, displaying a color correspondingto average color information for one screen of input image data at atime-associated position on the scale, displaying an image correspondingto the input image data, and displaying an index indicating a positioncorresponding to an imaging time of a designated image, so that it ispossible to visually and easily recognize how many and in which timeband designated images are present and easily determine organs from thecolors distinguished from one taken part from another one, thus makingit possible to easily recognize which part of which organ has moredesignated images.

Furthermore, according to the present invention, it is possible toprovide an image display method configured to have steps of displaying ascale indicating the overall imaging period of input image data taken intime sequence by an internal imaging device, discriminating organs basedon color information of one screen of input image data, displaying namesof the discriminated organ in association with the scale, displayingimages corresponding to the input image data and displaying an indexindicating the position corresponding to the imaging time of thedesignated image on the scale, so that organs in the body can easily bedetermined from the displayed organ names, whereby it is possible toeasily recognize which part of which organ has more designated images.

Moreover, according to the present invention, it is possible to providean image display program that allows a computer to execute processes ofdisplaying a scale indicating the overall imaging period of input imagedata taken in time sequence by an internal imaging device, displaying acolor corresponding to average color information for one screen of inputimage data at a time-associated position on the scale, displaying animage corresponding to the input image data, and displaying an indexindicating a position corresponding to an imaging time of a designatedimage, so that it is possible to visually and easily recognize how manyand in which time band designated images are present and easilydetermine organs from the colors distinguished from one taken part fromanother one, thus making it possible to easily recognize which part ofwhich organ has more designated images.

Furthermore, according to the present invention, it is possible toprovide an image display program that allows a computer to executeprocesses of displaying a scale indicating the overall imaging period ofinput image data taken in time sequence by an internal imaging device,discriminating organs based on color information of one screen of inputimage data, displaying names of the discriminated organ in associationwith the scale, displaying images corresponding to the input image dataand displaying an index indicating the position corresponding to theimaging time of the designated image on the scale, so that organs in thebody can easily be determined from the displayed organ names, whereby itis possible to easily recognize which part of which organ has moredesignated images.

Although the invention has been described with respect to a specificembodiment for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art which fairly fall within the basic teaching hereinset forth.

1. A method of displaying in-vivo image data, comprising: receivingimage data of a subject obtained by a swallowable in-vivo imagingdevice; displaying a graphical representation which represents the imagedata; and displaying along the graphical representation an indictoradapted to indicate a portion of the graphical representation.
 2. Themethod according to claim 1, wherein the portion corresponds to aportion of the digestive tract of the subject.
 3. The method accordingto claim 2, wherein the portion of the digestive tract of the subjectcomprises one or more of esophagus, stomach, small intestine and largeintestine.
 4. The method according to claim 1, wherein the graphicalrepresentation is adapted to indicate a magnitude of the image data. 5.The method according to claim 4, wherein the magnitude includes animaging period of the image data.
 6. The method according to claim 2,wherein the image data comprises a plurality of images and the graphicalrepresentation comprises a summary of a color representation of at leasta portion of the image data.
 7. The method according to claim 6, whereinthe graphical representation has a bar shape.
 8. The method according toclaim 7, wherein the graphical representation is adapted to indicate amagnitude of the image data.
 9. The method according to claim 7, whereinthe displaying of the graphical representation comprises displaying thegraphical representation in a substantially horizontal direction withrespect to a display area.
 10. The method according to claim 9, whereinthe image data includes a plurality of images and the method furthercomprising displaying at least some of the plurality of images in afirst area of the display area, wherein the displaying of the graphicalrepresentation comprises displaying the graphical representation in asecond area located below the first area.
 11. The method according toclaim 6, wherein the image data includes a plurality of images and thegraphical representation comprises a summary of a color representationof at least some of the plurality of images, the summary being obtainedby analyzing at least some of the plurality of images.
 12. The methodaccording to claim 11, wherein the summary contains an average color ofthe plurality of images.
 13. The method according to claim 1, whereinthe displaying of the indicator includes displaying the indicator suchthat an area of the graphical representation is defined by theindicator.